fix: match HF-style gemma-4 model names for tool_responses role#6340
fix: match HF-style gemma-4 model names for tool_responses role#6340abhiramArise wants to merge 4 commits into
Conversation
The gemma4 role fix (google#5650) only matched Ollama-style names like gemma4:e2b. HF/vLLM/llama.cpp use gemma-4 with a hyphen, e.g. google/gemma-4-26B-A4B, which the substring check missed, causing the wrong tool role on non-Ollama backends. Fixes google#6334
|
Hi @abhiramArise , Thank you for your contribution! We appreciate you taking the time to submit this pull request. Your PR has been received by the team and is currently under review. We will provide feedback as soon as we have an update to share. |
|
Hi @wyf7107 , can you please review this. LGTM. |
|
Looked into the CI failures — they appear unrelated to this PR's change (+30/-2, Mypy failures: point to Unit Test failures: all 12 failures are Ran the relevant test file locally and it passes cleanly: Let me know if there's anything you'd like me to adjust. |
The gemma4 role fix (#5650) only matched Ollama-style names like gemma4:e2b. HF/vLLM/llama.cpp use gemma-4 with a hyphen, e.g. google/gemma-4-26B-A4B, which the substring check missed, causing the wrong tool role on non-Ollama backends.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
1. Link to an existing issue (if applicable):
2. Or, if no issue exists, describe the change:
Problem:
When using ADK's LiteLLM integration with Gemma 4 models served via vLLM or llama.cpp (using Hugging Face's naming convention, e.g.
google/gemma-4-26B-A4B), tool call results are sent with roletoolinstead of the requiredtool_responses. This causes the model to hallucinate, produce garbage output, or loop until OOM. The existing fix in #5650 only checked for the substring"gemma4", which matches Ollama-style names (gemma4:e2b) but not the hyphenated Hugging Face naming (gemma-4).Solution:
Replaced the plain substring check with a regex-based helper,
_is_gemma4_model(), using the patterngemma-?4, so it matches bothgemma4andgemma-4naming conventions. Applied this helper at both call sites that previously used the substring check.Testing Plan
Unit Tests:
Added
test_gemma4_hf_style_naming_uses_tool_responses_roletotests/unittests/models/test_lite_llm_gemma_tool_role.py, verifying thatgoogle/gemma-4-26B-A4B(the exact model name from the bug report) resolves to roletool_responses.Ran the full test file locally:
python -m pytest tests/unittests/models/test_lite_llm_gemma_tool_role.py -v
======= 8 passed, 4 warnings in 8.33s =======
Manual End-to-End (E2E) Tests:
Verified via unit tests above that both Ollama-style (
gemma4:e2b) and Hugging Face-style (gemma-4-26B-A4B) model names correctly resolve totool_responses, and unrelated models still resolve totoolas before (no regression).Checklist
Additional context
N/A